Anytime Point Based Approximations for Interactive POMDPs

Partially observable Markov decision processes (POMDPs) have been largely accepted as a rich-framework for planning and control problems. In settings where multiple agents interact POMDPs prove to be inadequate. The interactive partially observable Markov decision process (I-POMDP) is a new paradigm that extends POMDPs to multiagent settings. The added complexity of this model due to the modeling of other agents beliefs and posible actions makes exact methods in this framework even harder to compute. Thus, a need arises for good approximation methods that could find solutions and in shorter periods of time than what has been developed so far. We develop a point based method for solving finitely nested interactive POMDPs approximately. The method mantains a set of belief points and form value functions including only the value vectors that are optimal at these belief points. Since I-POMDPs computation depends on the prediction of the actions of other agents in multiagent settings, an interactive generalization of the point based value iteration (PBVI) methods needed to be developed. We present some empirical results in domains on the literature, propose a new domain, and discuss computational savings. Index words: Markov Decision Process, Multiagent systems, Decision making, POMDP Anytime Point Based Approximations for Interactive POMDPs